An entropy-based measure of morphological information

Traditional approaches to morphology tend to treat inflectional systems not as unstructured sets of forms with shared stems or roots but as structured networks of elements. The interdependency of elements is, as Matthews (1991: 197) notes, ‘the basis of exemplary paradigms’ in the classical grammatical tradition. Although the exemplary patterns and leading forms of traditional descriptions bring out the structure of inflectional systems, traditional accounts are deficient – or at least incomplete – in a number of important respects. In particular, there is no method for measuring the implication structure of a set of forms or, no means of gauging the diagnostic value of specific forms within a set, and no generally accepted way even of identifying the leading forms of a system. The approach outlined in this talk proceeds from the observation that implicational structure involves a type of information, specifically information that forms within a set convey about other forms in that set. Information in this sense corresponds to reduction in uncertainty. The more informative a given form is about a set of forms, the less uncertainty there is about the other forms in the set. In inflectionally complex languages, a speaker who has not encountered all of the forms of a given item is faced with some amount of uncertainty in determining the unencountered forms. If the choice of each form were completely independent, the problem of deducing unencountered forms would reduce to the problem of learning the lexicon of an isolating language. However, in nearly all inflectional systems, there are at least some forms of an item that reduce uncertainty about the other forms of the item. Once these notions are construed in terms of uncertainty reduction, the problem of measuring implicational structure and diagnostic value is susceptible to well-established techniques of analysis. The uncertainty associated with the realization of a paradigm cell correlates with its entropy (Shannon 1948) and the entropy of a paradigm is the sum of the entropies of its cells. The implicational relation between a paradigm cell and a set of cells is modelled by conditional entropy, the amount of uncertainty about the realization of the set that remains once the realization of the cell is known. The diagnostic value of a paradigm cell correlates with the expected conditional entropy of the cell, the average uncertainty remains in the other cells once the realization of the cell is known.